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ABSTRACT 

Academic researchers have often touted the growing importance of “soft skills’’ for modern day 
business leaders, especially leadership and communication skills. Despite this growing interest 
and attention, relatively little work has been done to develop and validate tools to assess soft 
skills. Forty graduate students from nine MBA courses completed the McCann Soft Skills 
Assessment Tool - a paper-pencil test designed to measure leadership, teamwork, critical thinking, 
logical reasoning, communication, and holistic thinking. These students were also rated by their 
instructors on the same six dimensions in an attempt to validate the soft skills instrument. Results 
showed significant correlations betw’een leadership and communication assessment scores and 
faculty ratings of students on the same dimensions; however none of the other four soft skill 
dimensions were empirically validated. Data also indicated moderate test — retest reliability for 
the assessment tool. While some limitations are acknowledged in the current study, findings 
suggest that further refinement and empirical validation of the McCann Soft Skills Assessment 
Tool can better aid practitioners in understanding and developing these important dimensions of 
business competency. 

Keywords: Soft Skills Assessment; MBA Students; McCann Soft Skills Assessment Tool; Faculty Ratings of 

Students 


INTRODUCTION 

s one looks out upon the ever changing and increasingly complex global landscape of business and 
industry, it becomes more and more apparent that innovations and problem-solving will require 
multi-disciplinary teams engaging in highly intense collaboration. As such, the emphasis on greater 
knowledge and technical skills as a key asset, will need to be complemented with refined soft skills (specifically, 
leadership, teamwork, critical and holistic thinking, logical reasoning, and communication skills) to allow the proper 
expression, implementation, and collaboration for optimal use of knowledge assets. Chakraborty (2009) made a 
compelling argument for the essential role of soft skills in formulating well-rounded individuals who will be the key 
constituents in successfully navigating the change with which we are confronted. Those constituents begin with 
students and educators, as well as their institutions; but also extend to society, in general, with the vested interest in 
advancing progress in a manner “to revive the falling standards of values and ethics and also by incorporating the 
right mix of domain knowledge and soft skills quotient” (Chakraborty, 2009, p. 16). 

The call for greater attention to soft skills development has been trumpeted in a number of disciplines, 
particularly business management (Mitchell, Skinner, & White, 2010; Rangnekar, 2011) and information 
systems/technology (Beard, Schwieger, & Surendran, 2008; Joseph, Ang, Chang, & Slaughter, 2010). Collectively, 
these authors suggest that soft skills are greatly desired by employers, yet are elusive in many of these disciplines’ 
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contemporary models of pedagogy, and have not been sufficiently explored due to great difficulty in capturing them 
through systems of outcomes assessment. This last point may be the key to understanding where better solutions for 
nurturing soft skills within higher education begins - with a reliable and valid assessment tool. Not only is such a 
tool lacking within higher education teaching - learning cycles, but also in the training industry where a surge of e- 
learning platforms targeting the development of soft skills among trainees exists (Roberts, 2008). As the 
measurement challenge continues, the demand for soft skills persists. 

A recent survey of 276 prospective employers identified the top 12 attributes deemed most important to 
them (Koncz & Collins, 2007). Communication skills were at the top of the attributes with teamwork skills coming 
in at third. Interpersonal, problem-solving and analytic skills were in the middle of the pack capturing fifth through 
seventh place respectively. Although the prototypical dimension structure for soft skills varies widely, this 
employer list aligns extremely well with the six soft skill dimensions assessed in the current study, the McCann 
Business Soft Skills Assessment (Schultz, 2010). The underlying dimension of the McCann tool was founded upon 
the SCANS (Secretary's Commission on Achieving Necessary Skills; U.S. Department of Labor, 1990) standards, 
and includes: 

1. Leadership - The ability to effectively guide or direct others. 

2. Teamwork - The ability to cooperatively and with a coordinated effort work with others to reach a common 
cause or interest. 

3. Critical Thinking - Disciplined thinking that is clear, rational, open-minded, and informed by evidence 
which helps one to decipher whether a claim is true or false. 

4. Logical Reasoning - The ability to determine a conclusion, by applying a rule to a given precondition. The 
three types of logical reasoning include deduction, induction, and abduction. 

5. Communication Skills - the ability to convey thoughts, opinions, and information to others. This can 
include speech, writing, or signs. 

6. Holistic Thinking - The ability to recognize the larger picture and understand the interrelationship between 
the parts and the whole. 

Working in collaboration with several institutions of higher education, McCann Associates developed and 
conducted some initial pilot testing resulting in the 36 items instrument (six items per dimension) used in this study. 
Whereas other studies have typically employed self-report ratings to personalized statements (Kumara & 
Sahasranam, 2008), or based items on discipline-specific critical incidents (Joseph et ah, 2010), the McCann tool 
employs generic business scenario questions requiring a correct answer response. 

The purpose of the present study was to evaluate the developing tool, McCann Business Soft Skills 
Assessment Tool, in terms of its predictive validity of instructor ratings of students on its six dimensions: leadership, 
teamwork, critical thinking, logical reasoning, communication skills, and holistic thinking. 

METHOD 

Participants 

Instructors in nine MBA courses were encouraged to promote student participation in the study. A total of 
40 students completed the soft skills assessment tool, and 26 of those 40 completed it a second time to assess test- 
retest reliability. 

Materials 

Participants were asked to complete an on-line version of the 36 question McCann Business Soft Skills 
Assessment Tool. The instrument examined six domains (six items per domain) of soft skills: leadership, teamwork, 
critical thinking, logistic reasoning, communication skills, and holistic thinking. Two forms were used (test/re-test) 
and counterbalanced for order. 

Criterion data was collected from MBA faculty using a rating grid presented to them via SurveyMonkey. 
Instructors rated individual students on each dimension using the five-point scale indicated below. Grade 
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descriptors were based on typical distributions for course grades - with the top anchor, excellent, attempting to allow 
discrimination among top student achievers in the direction of the skew: 

• Poor (C+ or lower) 

• Below Average (B or B-) 

• Average (B+) 

• Above Average (A- or A) 

• Excellent (A+) 

Instructors were also permitted to record a Not Applicable option if they felt there was not enough 
behavioral evidence to provide a reliable rating. 

Procedure 

Participating graduate students completed and returned consent forms to class instructors who forwarded 
them to researchers. The consent forms also included permission for instructors to submit to the researcher 
confidential ratings of the student on the soft skill dimensions. Students then received an e-mail from the researcher 
with a link for the assessment, as well as a username and password to access the instrument. Study participants had 
one week to complete the online tool within a continuous 75 minute time limit. Demographic data were also 
collected online during this time. 

Students who consented to be rated by their instructors on the soft skills dimensions (100% consented) 
were listed in a separate e-mail sent to the instructor toward the end of the course. That e-mail also contained a link 
to the SurveyMonkey rating forms for each student. Instructors were given definitions of each dimension and 
directed to evaluate each student. 

Results 


The descriptive statistics for the overall McCann Soft Skills Assessment Tool scores, individual dimension 
scores, and criteria ratings are provided in Table 1. Average ratings tended to come in above the midpoint of the 
scale, but standard deviations did not indicate any concerns regarding lack of variability in the data. The minimum 
and maximum values also suggested that, for the most part, the full range of values were being employed within 
each dimension for both predictors (i.e., test scores) and criteria (i.e., instructor ratings). Based on the 26 subjects 
who were able to take both forms of the soft skills assessment tool, approximately six weeks apart, test - retest 
reliability was moderate in strength (r = 0.68, p < .01). Typically, a desirable reliability correlation should exceed 
the threshold of .80 (Dunn, 2001). All reported correlations are Pearson product moment correlation coefficients. 


Table 1: Descriptive Statistics for Predictor and Criteria Measures 


Measures 

11 

M 

SD 

Min 

Max 

Predictors (6 point scales) 






Overall Score 

40 

24.03 

4.08 

13 

31 

Leadership Score 

40 

4.43 

1.22 

2 

6 

Teamwork Score 

40 

4.05 

1.11 

1 

6 

Critical Thinking Score 

40 

3.98 

0.95 

2 

6 

Logical Reasoning Score 

40 

3.48 

1.28 

1 

6 

Communication Score 

40 

3.70 

0.97 

1 

5 

Holistic Thinking Score 

40 

4.35 

0.89 

2 

6 

Criteria (5 point scales) 






Leadership Rating 

27 

3.15 

0.86 

1 

4 

Teamwork Rating 

30 

3.53 

0.90 

1 

5 

Critical Thinking Rating 

31 

3.16 

1.07 

1 

5 

Logical Reasoning Rating 

30 

3.43 

0.97 

1 

5 

Communication Rating 

30 

3.37 

0.81 

1 

5 

Holistic Thinking Rating 

31 

3.39 

0.80 

2 

5 
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Predictive validity correlations for each predictor - criterion pair within each soft skill dimension are 
reported in Table 2. The two dimensions to yield significant correlations were leadership (r = 0.52, p < .01), and 
communication (r = 0.54, p < .01). The other dimensions did not yield significant correlation between predictor 
scores and instructor ratings. Although the correlations for teamwork (r = .25) and holistic thinking (r = .27) were 
moderately positive, the relationship was not strong enough to yield significance at the .05 probability level. 
Correlations within the critical thinking and logical reasoning dimensions were positive, but very small. To better 
understand the lack of consistency in significant predictive validity indicators across dimensions, the “soft skills” 
factor structure was analyzed. More specifically, the degree of independence of each soft skill dimension (within 
assessment scores and within instructor ratings) was evaluated to investigate the degree to which each measure is 
able to uniquely distinguish between each of the six soft skill dimensions. 


Table 2: Predictive Validity Correlations between Predictor Scores from the Soft Skills 
Assessment and Criterion Ratings from Course Instructors for each Dimension 


Dimension 

Correlation 

1. Leadership 

.52** 

2. Teamwork 

.25 

3. Critical Thinking 

.07 

4. Logical Reasoning 

.14 

5. Communication 

.54** 

6 . Holistic Thinking 

.27 


Note. ** p < .01 


It should be noted, however, that some relatedness between dimensions is expected since they share an 
underlying commonality reflected in their classification under the “soft skills” umbrella. Similar to Sternberg’s “g 
factor” (general intelligence) often explored in cognitive psychology as an overarching trait underlying different 
types of intelligence (Sternberg, 2003), there is possibly a similar general personality/social trait in the soft skills 
arena. It may also be that this general soft skills trait similarly drives correlations between distinctive measures of 
soft skills aptitude in the way that the g factor drives small to moderate correlations in the cognitive aptitude space. 
Recent work exploring co-curricular assessment scale development, suggests that perhaps “emotional intelligence” 
may be the underlying factor (Feldman, Aper, & Meredith, 2011). Regardless, to merit effort of quantification, 
measures of sound psychometric quality should be sufficiently sensitive to reliably distinguish levels of each 
dimension within individual subjects. Inordinately high correlations between dimensions (i.e., dimensions not being 
independent of each other) would be a concerning indicator about the measurement source’s inability to assess 
dimensions independently. 

The results of inter-dimensional correlations for both sources of soft skill measurements are presented in 
Table 3. It is clear that the McCann Soft Skills Assessment Tool does a fairly good job at yielding profiles in which 
dimensions tend to be generally independent of each other. Only three of the fifteen dimensional pairings (i.e., 
leadership - critical thinking, teamwork - communication and communication - holistic thinking) yield a significant 
pattern of interrelated scores. This pattern certainly suggests a keener sensitivity to distinguishing between 
dimensions relative to the pattern that emerges for instructor ratings. Within the instructor ratings of student soft 
skills, the same analyses indicate not only a high correlation for all fifteen pairs of dimensions, but fourteen of 
fifteen are significant at the .01 probability level. These high correlation results suggest a commendation for the 
psychometric properties of the soft skills assessment tool, and may also shed some light on the inconsistencies in the 
predictive validity outcomes in Table 2 as discussed in the following section. 


Table 3: Inter-Dimension Correlations for Both the Predictor Scores (Top Triangle) and 
__ Ratings from Course Instructors (Bottom Triangle) _ 


Dimension 

1 

2 

3 

4 

5 

6 

1. Leadership 


.23 

.01 

.34* 

.31 

-.02 

2. Teamwork 

.85** 


.20 

.22 

.42** 

.27 

3. Critical Thinking 

79 ** 

.64** 


.20 

.19 

.13 

4. Logical Reasoning 

72** 

.56** 

.81** 


.22 

.23 

5. Communication 

70** 

.44* 

.82** 

.65** 


.48** 

6 . Holistic Thinking 

.60** 

49 ** 

. 74 ** 

7g** 

. 66 ** 



Note. ** p < .01; * p < .05 
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Discussion 

Based on (a) test-retest reliability, (b) the pattern of validity correlations particularly for leadership and 
communication, (c) the sensitivity in distinguishing respondents (see standard deviations and minimum/maximum 
values in Table 1), and (d) predominantly independent sub-scores for the predictor dimensions (see Table 3), this 
pilot study suggests that the McCann Soft Skills Assessment Tool is useful. 

Predictive Validity 

The two dimensions identified by many as the most important of the soft skills, leadership and 
communication, worked particularly well to predict instructor ratings of those students’ course performance in those 
dimensions. It has been asserted by experts in the field of business education that these two soft skill traits may be 
the most important ones to develop (Mitchell, Skinner, & White, 2010). Leadership and communication were 
identified by a sample of business educators as "the most important soft skills for success in the twenty-first century 
workforce” based on survey results (Mitchell, Skinner, & White, 2010, p. 52). On the criterion side, these may also 
be the most reliable ratings (compared to other dimensions) since instructors may have greater opportunity of 
exposure to such skills by their students, as well as a more concrete frame of reference for the behaviors that 
constitute these skills. Leadership and communication can be defined and observed in more behavioral terms; 
whereas logical reasoning, critical thinking, and holistic thinking are more mental/cognitively based. These skills 
may tend to require a more challenging level of inference and/or more in-depth exposure (to student thought 
processes) on the part of an evaluator. Although teamwork, like leadership and communication, has a strong 
behavior-based definition as well, actual team-based experiences may not have been formally implemented or 
explicitly demonstrated within the courses sampled. 

Factor Independence among Soft Skill Dimensions 

The McCann Soft Skills Assessment Tool was much more successful at measuring individual soft skills 
independent of the other dimensions compared to instructor ratings. Comparing the patterns of inter-dimensional 
correlations (Table 3) suggests that a “paper-pencil” skills assessment tool may enable a measurement strategy that 
can isolate a demonstration of the specific soft skill competency without the potential problems that can come from 
human judgment. The extremely large inter-dimensional correlations within the predictor criteria ratings suggest 
clear evidence of a “halo bias” contaminant (the strong influence by the instructor’s general impression of the 
student), inhibiting distinction between impressions of the student on each of the dimensions. Excessively high 
correlations among same source rating dimensions have been well established as a reliable indicator of halo bias 
(Fisicaro & Vance, 1994). 

In addition, halo bias may be a partial explanation for the inconsistent predictive validities listed in Table 1. 
If, as suggested by the business educators sampled by Mitchell, Skinner, and White (2010), there is an intuitive 
sense of importance for communication and leadership as the perceived dominant soft skill competencies, evaluation 
of students on these two dimensions may be trickling into effect and distort ratings in the other dimensions. This 
possibility is further supported by the aforementioned assertions about the greater availability of evaluative 
information for these two dimensions, leadership and communication in a typical classroom environment. 

Shortcomings and Limitations 

The current study has limitations that should be noted. First, employing a larger sample size would provide 
a more powerful study of reliability and predictive validity. Second, subject motivation was hard to assess given 
that it was an independent on-line administration. However, some concern persists since it was impossible to 
standardize how the assessment opportunity was introduced and embedded into the course as suggestions for 
bolstering student motivation and implementation into the course grading structure were left up to the cooperating 
instructors. 

The third, but most significant, concern lies with the large and consistent inter-correlation of (instructor) 
ratings of students suggesting some possible contamination of the criteria measures based on halo bias. Although 
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research has indicated that some “true halo” may exist (i.e., in contrast to halo bias) given common inter-relatedness 
of the actual performance dimensions (Goffin, Jelley, & Wagner, 2003), the inter-correlations demonstrated among 
performance ratings by instructors seem extreme in this study. 

Future Directions 

To realize greater potential of the McCann tool, and despite concerns with current criteria measures, 
evidence suggests that further piloting would be worthwhile, as well as some possible guidance on which 
dimensions may need refinement. In particular, holistic thinking tended to have a high average and low standard 
deviation among respondents relative to the other dimensions collectively. This may suggest some concern for the 
dimension’s sensitivity toward individual differences on that construct. Also, teamwork seems to remain an elusive 
construct to capture in the context of a quantitative inventory as opposed to more traditional behavior-based ratings 
(Dickinson & McIntyre, 1997), although the uncertain accuracy of instructor ratings on teamwork leaves this 
unclear. 


Future development and refinement should include validation with more authentic sources of “soft skill” 
student learning outcomes (Banta, Lund, Black, & Oblander, 1996; Janesick, 2006). Future studies should also 
consider more sophisticated rubrics to guide subjective rating systems (Beard et ah, 2008). Authentic assessments 
have been defined as “the real application of a skill beyond its instructional context . . . requiring students to apply 
what they have learned” (Mertler, 2003, pp. 10-11). This learning typically takes place in a real world type of 
scenario. Authentic course-embedded assessments are very difficult and time consuming to implement, but their 
cost-benefits should be carefully weighed in terms of assessment system quality and pedagogical improvement 
potential. 

CONCLUSION 

Still an evolving area of study, many myths exist that oversimplify soft skills in terms narrowing their 
multifaceted nature and entrenching them as purely inborn traits (Rao, 2012). A measure like the McCann Soft 
Skills Assessment Tool can go a long way to debunking those myths and properly illustrating these important 
business competencies. As mentioned, there is also a growing call for greater attention to soft skills development in 
academia and global industry which further drives the need for web-based and e-learning platforms with sound 
pedagogical practices to advance soft skills (Morgan & Adams, 2009). As with any sound teaching-learning 
dynamic, systems will require reliable and valid assessments to provide formative and summative feedback for 
ongoing improvement and standards accountability. A quantitative objective assessment of soft skills will be a 
valuable tool for these advances, and evidence suggests the McCann Soft Skills Assessment Tool has the potential to 
eventually serve this purpose. 
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